Math - Stat - 491 - Fall 2014 - Notes - IV

We have so far proved a number of equivalent characteristic properties of transient and recurrent states. Theorem 2.1. A state x in a finite Markov chain is transient iff it satisfies the following equivalent conditions: 1. The probability ρxx of returning to x, is less than 1. 2. Starting at x the expected number of returns to x, Ex(Nx) is finite. 3. In the Markov chain graph there is a directed path from x to some recurrent node y but no return directed path. 4. For some state y, the probability of reaching in finite time from x to y is nonzero but the probability of reaching in finite time from y to x is less than 1. 5. If π(·), is a stationary distribution of the Markov chain π(x) = 0. (Note: π(x) = 0 in every stationary distribution, for a transient state. If there is more than one stationary distribution, every recurrent state will have value zero in some of them, but not in all.) Sketch of proof: (1) is the definition. (2) is equivalent to (1) because Ex(Nx) = ρxx/(1− ρxx). (2→ 3) Let C be the collection of all states reachable from x by a directed path. We then have Σy∈CEx(Ny) = Σy∈CΣ ∞ n p (x, y) = Σn Σy∈Cp (x, y) = Σn (1) =∞.